Generalization and Diversity in Co-evolutionary Learning

Games have long played an important role in the development and further understanding of co-evolutionary learning systems. In particular, the search process in co-evolutionary learning is guided by strategic interactions between solutions in the population, which can be naturally framed as game-playing. We study two important issues in co-evolutionary learning–generalization performance and diversity–using games. On the one hand, one is concerned with the co-evolutionary learning of strategies with high generalization performance, that is, strategies that can outperform against a large number of test strategies (opponents) that may not be seen before during co-evolution. On the other hand, the other one is concerned with diversity levels in the population that may lead to the search of strategies with poor generalization performance. It is not known if there is a relationship between generalization and diversity in co-evolutionary learning. This paper investigates whether there is such a relationship in co-evolutionary learning through a detailed empirical study. We systematically investigate the impact of various diversity maintenance approaches on the generalization performance of co-evolutionary learning quantitatively using case studies. The problem of the iterated prisoner’s dilemma (IPD) game is considered. Unlike what was done in past studies, we can measure both the generalization performance and the diversity level of the population of evolved strategies. Results from our case studies show that the introduction and maintenance of diversity do not necessarily lead to the co-evolutionary learning of strategies with high generalization performance. However, if individual strategies can be combined in the form of ensembles, there is the potential of exploiting diversity in co-evolutionary learning to improve generalization performance. Specifically, when the introduction and maintenance of diversity lead to a speciated population during co-evolution, where each specialist strategy is capable of outperforming different opponents, the ensemble can have a significantly higher generalization performance compared to individual strategies when it can sufficiently often choose the best specialist strategy against a test strategy.